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ABSTRACT 

This study compared ISTEP+ {Indiana Statewide Testing for 
Educational Progress) writing test scores before the implementation of a 
writing improvement program with ISTEP+ writing test scores. The study was 
conducted two years after such a program was implemented. Data from two 
groups was collected for this comparison. The first group used was the 1999- 
2000 North Vermillion Elementary 3rd grade ISTEP+ writing test scores. This 
group of students did not participate in any writing improvement program. The 
second group used was the 2001-2002 North Vermillion Elementary 3rd grade 
ISTEP+ writing test scores. This second group of students participated for 
two years {prior to taking ISTEP+) in a school-wide writing improvement 
program. It was concluded that there was a significant difference in ISTEP+ 
writing test scores at the 0.05 level of significance. It can be further 
concluded that the writing improvement program did have a significant effect 
on ISTEP+ writing test scores. Contains 7 references, and 2 charts and 2 
tables of data. {Author/RS) 
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Abstroct 



This study compared ISTEP+ writing test scores before the implementation 
of a writing improvement program with ISTEP+ writing test scores. The study was 
conducted two years after such a program was implemented. Data from two groups 
was collected for this comparison. The first group used was the 1999-2000 North 
Vermillion Elementary 3*^ grade ISTEP+ writing test scores. This group of students 
did not participate in any writing improvement program. The second group used was 
the 2001-2002 North Vermillion Elementary 3*^ grade ISTEP+ writing test scores. 
This second group of students participated for two years (prior to taking ISTEP+) 
in a school wide writing improvement program. It was concluded that there was a 
significant difference in ISTEP+ writing test scores at the 0.05 level of 
significance. Therefore, the null hypothesis must be rejected. It can be further 
concluded that the writing improvement program did have a significant effect on 
ISTEP+ writing test scores. 
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Background of the Problem 

Student achievement in core subjects has been a major focus in many 
schools nationwide and writing is often identified as an area that heeds 
improvement. The NAEP (National Assessment of Educational Progress) test is 
given every two years to students aged 9, 13, and 17. Between 1992 and 1994, 
Campbell, Reese, O'Sullivan, and Dossey (1994) reported that NAEP writing test 
scores fell in all three grade levels and that scores for grades 8 and 11 yielded even 
lower results than when the tests were first administered ten years prior. 

In recent years, much emphasis has been placed on achievement test scores 
and the impact they can have on school accreditation status. Schools are required 
to strictly follow state standards in all grade levels and are held accountable for 
improvements in student performance. School corporations nationwide have 
implemented writing improvement plans in an attempt to improve lackluster writing 
test scores. 

To better facilitate the implementation of a writing improvement program, 
Cowell and Butler (1984) reported that teachers from River Mill Elementary School 
in Estacada, Oregon attended workshops to learn assessment procedures, state 
rubrics, and the steps of the writing process for each grade level. In addition, 
grade level teachers brainstormed topics to use for classroom writing prompts and 
other writing related ideas, such as student journals and published student books. 
Samples of student writing were also used for scoring practice, to allow teachers to 
better understand how to help students improve their own writing. 

Some programs, such as the one used at River Mill Elementary School in 
Estacada, Oregon (Cowell et al. 1984), include in-depth instruction to help students 
learn the steps of the writing process, scheduled writing prompts for all grade 
levels, and the assessment of writing assignments using rubrics. Educators must 
walk a fine line when implementing this type of intervention, since criticism of 
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"teaching the test" is a common response. New Hampshire Superintendent Champlin 
(Willett, 2001) answered this charge when questioned about his school's tutoring 
sessions, aimed at improving writing test scores. "We're not teaching the to the 
test. We're simply tryirg to help students improve their writing ability. We want 
our students to be able to communicate articulately through writing." 

In contrast to this point of view, Hegarty (2000) reported that some Florida 
schools, like A. A. Dixon Elementary in Pensacola, FL, direct their instruction to 
follow the Florida Writes test. This particular test, introduced in 1993, uses a 
student writing prompt and a 45-minute writing period. The "prompts" are scored 
on a scale of 1 - 6, with 6 being the highest score. Teachers expressed that 
"practice, practice, practice," as well as "teaching tailored to the statewide test" 
has given their students a positive advantage. Principal Judith Ladner of A. A. Dixon 
Elementary comments, "It gets to be like breathing. Constant writing. Every day." 
Her school, along with other "F-rated" Florida schools, improved their state writing 
test scores tremendously after implementing a writing improvement program. In 
1993, 20% of fourth grade students in these schools scored a 3.0 or higher on the 
state writing test. By 1996, 40% of fourth graders improved their scores and 
in 2000, 84% of fourth grade students Scored a 3.0 or higher on the Florida Writes 
test. 

Logically, the more students are involved in their own writirg experiences, 
the more enthusiastic they will be. Eve Bearne (2000) suggested, "Setting children 
to write their own books and then to reflect on the process of writing" is a good 
way to bring writing into school curriculums. Bearne added, "One way of improving 
children's writing is to get them to talk about their work." 

Another approach involves using parents and peers to assist the writer 
during the pre-writing and draft Stages. According to Topping, Nixon, Sutherland 
and Yarrow (2CX)0), this method, known as paired writing, encourages the writer to 
read his/her piece aloud and to edit the work to the final copy stage with the help 
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of an adult or peer. This method allows students to benefit from both a structured 
and collaborative approach, so that "writing is increasingly seen as a social rather 
than a solitary activity." 

In Indiana, many schools have not only been confronted with low test scores, 
but have also been bombarded with PBA (Performance Based Accreditation) 
guidelines and must show progress in selected areas in order to maintain an 
accredited school status. To accomplish this task some corporations, like the 
Monroe County Community Schools, (Adams, 20CX)) have enlisted the help of grant 
monies and initiated programs to improve student test scores in the areo of writing. 
Staff development, with the help of a professional consultant, the use of rubrics, 
and a school wide writing program are all included in these intervention strategies. 

This approach was similar to that used at River Mill Elementary School in 
Estacada, Oregon, where Cowell et al. (1984) reported the use of a consultant to 
assist teachers in learning the "holistic" writing approach. This program "requires 
teachers, students, and evaluators to consider a piece of writing as a whole — a 
single, integrated, unique expression, rather than as a collection of parts to be 
subjected to a separate analysis and evaluation for spelling, grammatical usage, 
construction, etc." This method also requires a subjective assessment to critique 
whether each piece of writing is "competent" or not. Competency is scored on a 
scale of 1 - 4, with 4 being the highest mark. This writing model yielded an increase 
from 26% of students passing the Estacada Writing Assessment Test, to 54% of 
students passing the test, after only one year of use. 

It seems that writing competency has become a focal point for many schools 
across the nation. Numerous state tests and assessments now include a strong 
writing component that can weigh heavily on the total reading/language student 
score. To compound this issue, some schools are under pressure to show 
improvements in writing in order to receive both a favorable rating from the state. 
To accomplish this task, writing improvement programs of various kinds have been 
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implemented at every grade level end many include the use of writing prompts, 
scoring rubrics, and instruction in the steps of the writirg process. With all of 
these interventions in place, improved writing test scores would seem to be the 
obvious result. If these plans are well thought out and executed by teachers end 
administrators, then the effect on student writing test scores should be a positive 
one and in time, these endeavors will prove to be worthwhile. 
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Statement of the Problem 

In recent years, many Indiana schools have found themselves facing the 
reality of low ISTEP+ test scores. One subject area that has received much 
attention is writing. Pressure from the state to show improvement in these scores, 
has lead Some schools to the question, "Can low achievement scores in writing 
be improved?" 

North Vermillion Elementary School faced a similar dilemma two years ago 
and was urged by the state to devise a way to improve low student writing test 
scores. A school wide writir^ improvement program was chosen os a possible 
solution. Does strict adherence to this type of intervention, which included 
teachir^ the steps of the writing process, the use of practice writing prompts, and 
the use of state rubrics for assessment, help students improve writing test scores? 

For the purpose of this study, a null hypothesis was tested: There is no 
difference between the test scores of students who participated in the writing 
improvement program and the test scores of students who did not participate in 
the writing improvement program. 
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AAethodoloov 

The sample used for this study was taken from North Vermillion Elementary, 
which is a consolidated, rural school, and the only elementary in that corporation. 
The socioeconomic status of students in this area ranges from low to middle class, 
which is typical of many smaller schools in Western Indiana. Northern Vermillion 
County has a slightly diverse population, although most people are Caucasian. This 
sort of population is indicative of other rural schools in this geographic area. 

Two years ago. North Vermillion Elementary implemented a school wide 
program for improving writing test scores. Teachers received both training and 
guidance in planning a writing curriculum for each grade level, which followed state 
standards. In-service workshops were held to allow staff to familiarize themselves 
with the State writing rubrics and assessment procedures. “Hallway walks" were 
used to give teachers a chance to observe other Staff members durir^ instructional 
writing times, and to gather information and/or ideas for use in their own 
classrooms. This plan also included in-depth classroom instruction to help students 
learn the steps of the writing process, 3 scheduled writing prompts per school year 
for all grade levels, and the assessment of selected writir^ assignments using state 
rubrics. In addition, daily writing times, published student books, and student 
writing journals were used to encourage interest and improvement in writing. Data, 
in the form of ISTEP+ writing test scores, was gathered from the 1999-2CXX) and 
the 2001-2002, 3 '^ grade classes. The 1999-2000 classes had a total enrollment of 
69 students, from 3 classrooms, which did not participate in any writing 
improvement program. The 2001-2002, classes had a total enrollment of 63 
students, from 3 classrooms, who had participated in a writing improvement 
program for two years prior to taking the ISTEP+ test. 

A random sample of ISTEP+ writing test scores, ranging from a low of 1 to a 
high of 6 possible points was taken from the total class summary report, from both 
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the 1999-2000 and 2001-2002 classes. This sample included every third score on 
the list, starting alphabetically from the bottom up. A total of 23 scores was 
gathered for each group used in the study. Two additional scores were collected 
from the 2001-2002 classes, by using the first and last score on the alphabetical 
class list. This was done so that both groups would have the same amount (23) of 
test scores. 

An unpaired t test with a two-tailed P value was used to check for statistical 
significance. 





Results 



Tabic I 



ISTEP+ Writing Test Scores 


Classes 


Mean 


Standard Deviation 


1999- 2000 


3.22 


0.736 


2001-2002 


2.83 


0.491 



Tabic I shows a mean score of 3.22 for the 1999-2CXX) classes, as compared 
to a lower mean score of 2.83 for the 2001-2CX)2 classes. A standard deviation of 
0.736 was found for the 1999-2000 classes, while the 2001-2002 classes showed a 
standard deviation of only 0.491. These results were found after conducting an 
unpaired t test. 



Table 2 



Statistical Test Results 


t 


df 


p 


2.12 


44 


0.04 



Table 2 shows a t value of 2.12 with 44 degrees of freedom. The two-tailed 
probability of this result equals 0.04, which is statistically significant at the 0.05 
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ISTEP+ Writing Scores 
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Chart 1 shows the mean of group 1 (1999-2000) as 3.22 and the mean of group 2 
(2001-2002) as 2.83. The standard deviation for group 1 is 0.736 and the standard 
deviation for group 2 is 0.491. 




Chart 2 




Chart 2 breaks down the range of point totals for each group of the 3*^ grade test 
scores used in the study. Some students in the 1999-2000 group did score somewhat 
higher than the 2001-2002 group, with a point total of 5 out of a possible 6 points. 
However, some students in the 1999-2000 group also scored somewhat lower than the 2001- 
2002 group with a point total of 1 out of a possible 6 points. This chart also shows a modest 
gain at the low end of the point total scale, as well as a slight loss on the high end of the 
point total scale for the 2001-2002 group, as compared to the point totals of the 
1999-2000 group. 
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Discussions. Conclusions, and Recommendations 

This study examined the effectiveness of a writing improvement program at 
North Vermillion Elementary School. Sroup 1 (1999-2000) did not participate in any 
writing improvement program, while group 2 (2001-2002) did participate in a school 
wide writing improvement program for two years prior to taking the ISTEP+ test. 
ISTEP+ writing test scores from these two groups of 3'''' grade classes were 
compared to ascertain if any difference in test scores was evident after the 
implementation of that program. 

Statistical tests revealed lower mean test scores for group 2 (2001-2002) 
after participating in the writing improvement program for two years. A computed 
probability value of 0.04 was also found. This difference is statistically significant 
at the critical 0.05 level of significance, and the null hypothesis: there is no 
difference between the test scores of students who participated in the writing 
improvement and the test scores of students who did not participate in the writing 
improvement program, must be rejected. 

Possible explanations for this outcome include: 

1. The populations used in the Study were congruent, indicating that 
there was no difference between them. Higher mean scores in one group and lower 
mean scores in the other were acquired by chance. 

2. The populations were actually different; therefore the mean 
scores for each group were different and most likely this discrepancy was not 
caused by random sampling or chance. 

3. One group was stronger academically than the other group; 
therefore, the test scores would reflect that result, with or without the help of a 
writing improvement program. 

4. The amount of instruction and teaching styles used by teachers 
was different for each group, which might result in different writing test scores. 
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Looking back at Chart 2 (Range of ISTEP+ Writing Scores), the 2001-2002 
classes did make gains in writing test scores at the low end of the scale, even 
though they did not score as high over-all, as compared to the 1999-2000 classes. 
This indicates that the lower achieving students in the 2001-2002 group did 
benefit from the writing improvement program; even though the statistical test 
revealed that this group had lower scores over-all than the 1999-2000 group. Many 
times, the amount of instruction and additional attention given to certain students, 
be it the high, medium, or low achievers, leads to academic improvement for that 
group. In all likelihood, the lower skill students in the 2001-2002 group benefited 
from the new writing improvement program for that reason. 

This particular program at North Vermillion Elementary was implemented 
just two years ago. Such interventions must be given time to reach all students at 
all achievement levels. Perhaps in time, greater results in writing achievement test 
scores are possible for all skill levels and all students, with the assistance of a 
writing improvement program. 
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